Security News
ESLint is Now Language-Agnostic: Linting JSON, Markdown, and Beyond
ESLint has added JSON and Markdown linting support with new officially-supported plugins, expanding its versatility beyond JavaScript.
The unified npm package is an interface for parsing, inspecting, transforming, and serializing content through syntax trees. It is built on the concept of syntax trees and is often used to work with content in markdown, HTML, and plain text formats. Unified is part of the unified collective which provides a range of plugins and utilities for content processing.
Parsing Markdown to Syntax Trees
This feature allows you to parse Markdown content into an abstract syntax tree (AST) using the remark-parse plugin.
const unified = require('unified');
const markdown = require('remark-parse');
const processor = unified().use(markdown);
const tree = processor.parse('# Hello world');
console.log(tree);
Transforming Syntax Trees
This feature demonstrates transforming a Markdown AST to an HTML AST and then serializing it to an HTML string.
const unified = require('unified');
const markdown = require('remark-parse');
const remark2rehype = require('remark-rehype');
const html = require('rehype-stringify');
const processor = unified()
.use(markdown)
.use(remark2rehype)
.use(html);
const file = processor.processSync('# Hello world');
console.log(String(file));
Linting and Validating Markdown
This feature shows how to use unified with remark-lint to lint and validate Markdown content.
const unified = require('unified');
const markdown = require('remark-parse');
const remarkLint = require('remark-lint');
const processor = unified()
.use(markdown)
.use(remarkLint);
processor.process('# Hello world', function (err, file) {
console.error(report(err || file));
});
Compiling Markdown to HTML
This feature illustrates compiling Markdown to a fully formatted HTML document.
const unified = require('unified');
const markdown = require('remark-parse');
const remark2rehype = require('remark-rehype');
const doc = require('rehype-document');
const format = require('rehype-format');
const html = require('rehype-stringify');
const processor = unified()
.use(markdown)
.use(remark2rehype)
.use(doc)
.use(format)
.use(html);
const file = processor.processSync('# Hello world');
console.log(String(file));
markdown-it is a Markdown parser with a focus on speed and extensibility. It is similar to unified in that it can parse and render Markdown, but it does not use an AST or provide the same plugin ecosystem.
remarkable is another Markdown parser and renderer. It offers a similar feature set to markdown-it but also does not use an AST or have the extensive plugin system that unified offers.
showdown is a JavaScript Markdown to HTML converter. It is similar to unified in that it converts Markdown to HTML, but it does not provide a unified interface for parsing, transforming, and serializing content.
unified is an interface for processing text using syntax trees. It’s what powers remark, retext, and rehype, but it also allows for processing between multiple syntaxes.
The website for unified, unifiedjs.github.io
, provides a less
technical and more practical introduction to unified. Make sure to visit it
and try its introductory Guides.
npm:
npm install unified
var unified = require('unified')
var markdown = require('remark-parse')
var remark2rehype = require('remark-rehype')
var doc = require('rehype-document')
var format = require('rehype-format')
var html = require('rehype-stringify')
var report = require('vfile-reporter')
unified()
.use(markdown)
.use(remark2rehype)
.use(doc)
.use(format)
.use(html)
.process('# Hello world!', function(err, file) {
console.error(report(err || file))
console.log(String(file))
})
Yields:
no issues found
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
<h1>Hello world!</h1>
</body>
</html>
unified is an interface for processing text using syntax trees. Syntax trees are a representation understandable to programs. Those programs, called plugins, take these trees and modify them, amongst other things. To get to the syntax tree from input text there’s a parser. To get from that back to text there’s a compiler. This is the process of a processor.
| ....................... process() ......................... |
| ......... parse() ..... | run() | ..... stringify() ....... |
+--------+ +----------+
Input ->- | Parser | ->- Syntax Tree ->- | Compiler | ->- Output
+--------+ | +----------+
X
|
+--------------+
| Transformers |
+--------------+
Every processor implements another processor. To create a new processor invoke another processor. This creates a processor that is configured to function the same as its ancestor. But when the descendant processor is configured in the future it does not affect the ancestral processor.
When processors are exposed from a module (for example, unified itself) they should not be configured directly, as that would change their behaviour for all module users. Those processors are frozen and they should be invoked to create a new processor before they are used.
The syntax trees used in unified are Unist nodes: plain
JavaScript objects with a type
property. The semantics of those type
s are
defined by other projects.
There are several utilities for working with these nodes.
The following projects process different syntax trees. They parse text to their respective syntax tree and they compile their syntax trees back to text. These processors can be used as-is, or their parsers and compilers can be mixed and matched with unified and other plugins to process between different syntaxes.
The below plugins work with unified, unrelated to what flavour the syntax tree is in:
unified-diff
— Ignore messages for unchanged lines in TravisSee remark, rehype, and retext for lists of their plugins.
When processing documents metadata is often gathered about that document. VFile is a virtual file format which stores data and handles metadata and messages for unified and its plugins.
There are several utilities for working with these files.
To configure a processor invoke its use
method, supply it a
plugin, and optionally settings.
unified can integrate with the file-system through
unified-engine
. On top of that, CLI apps can be created with
unified-args
, Gulp plugins with unified-engine-gulp
, and
Atom Linters with unified-engine-atom
.
A streaming interface is provided through unified-stream
.
The API gives access to processing metadata (such as lint messages) and supports multiple passed through files:
var unified = require('unified')
var markdown = require('remark-parse')
var styleGuide = require('remark-preset-lint-markdown-style-guide')
var remark2retext = require('remark-retext')
var english = require('retext-english')
var equality = require('retext-equality')
var remark2rehype = require('remark-rehype')
var html = require('rehype-stringify')
var report = require('vfile-reporter')
unified()
.use(markdown)
.use(styleGuide)
.use(
remark2retext,
unified()
.use(english)
.use(equality)
)
.use(remark2rehype)
.use(html)
.process('*Emphasis* and _importance_, you guys!', function(err, file) {
console.error(report(err || file))
console.log(String(file))
})
Yields:
1:16-1:28 warning Emphasis should use `*` as a marker emphasis-marker remark-lint
1:34-1:38 warning `guys` may be insensitive, use `people`, `persons`, `folks` instead gals-men retext-equality
⚠ 2 warnings
<p><em>Emphasis</em> and <em>importance</em>, you guys!</p>
The processors can be combined in two modes.
Bridge mode transforms the syntax tree from one flavour (the origin) to another (the destination). Then, transformations are applied on that tree. Finally, the origin processor continues transforming the original syntax tree.
Mutate mode also transforms the syntax tree from one flavour to another. But then the origin processor continues transforming the destination syntax tree.
In the previous example (“Programming interface”), remark-retext
is used in
bridge mode: the origin syntax tree is kept after retext is done; whereas
remark-rehype
is used in mutate mode: it sets a new syntax tree and discards
the original.
processor()
Object describing how to process text.
Function
— New unfrozen processor which is configured to
function the same as its ancestor. But when the descendant processor is
configured in the future it does not affect the ancestral processor.
The following example shows how a new processor can be created (from the remark processor) and linked to stdin(4) and stdout(4).
var remark = require('remark')
var concat = require('concat-stream')
process.stdin.pipe(concat(onconcat))
function onconcat(buf) {
var doc = remark()
.processSync(buf)
.toString()
process.stdout.write(doc)
}
processor.use(plugin[, options])
Configure the processor to use a plugin and optionally configure that plugin with options.
processor.use(plugin[, options])
processor.use(preset)
processor.use(list)
plugin
(Plugin
)options
(*
, optional) — Configuration for plugin
preset
(Object
) — Object with an optional plugins
(set to list
),
and/or an optional settings
objectlist
(Array
) — List of plugins, presets, and pairs (plugin
and
options
in an array)processor
— The processor on which use
is invoked.
use
cannot be called on frozen processors. Invoke the processor
first to create a new unfrozen processor.
There are many ways to pass plugins to .use()
. The below example gives an
overview.
var unified = require('unified')
unified()
// Plugin with options:
.use(plugin, {})
// Plugins:
.use([plugin, pluginB])
// Two plugins, the second with options:
.use([plugin, [pluginB, {}]])
// Preset with plugins and settings:
.use({plugins: [plugin, [pluginB, {}]], settings: {position: false}})
// Settings only:
.use({settings: {position: false}})
function plugin() {}
function pluginB() {}
processor.parse(file|value)
Parse text to a syntax tree.
file
(VFile
)
— Or anything which can be given to vfile()
Node
— Syntax tree representation of input.
parse
freezes the processor if not already frozen.
processor.Parser
Function handling the parsing of text to a syntax tree. Used in the
parse phase in the process and invoked with a string
and
VFile
representation of the document to parse.
Parser
can be a normal function in which case it must return a
Node
: the syntax tree representation of the given file.
Parser
can also be a constructor function (a function with keys in its
prototype
) in which case it’s invoked with new
. Instances must have a
parse
method which is invoked without arguments and must return a
Node
.
processor.stringify(node[, file])
Compile a syntax tree to text.
string
— String representation of the syntax tree file.
stringify
freezes the processor if not already frozen.
processor.Compiler
Function handling the compilation of syntax tree to a text. Used in the
stringify phase in the process and invoked with a
Node
and VFile
representation of the document to stringify.
Compiler
can be a normal function in which case it must return a string
:
the text representation of the given syntax tree.
Compiler
can also be a constructor function (a function with keys in its
prototype
) in which case it’s invoked with new
. Instances must have a
compile
method which is invoked without arguments and must return a string
.
processor.run(node[, file][, done])
Transform a syntax tree by applying plugins to it.
node
(Node
)file
(VFile
, optional)
— Or anything which can be given to vfile()
done
(Function
, optional)Promise
if done
is not given. Rejected with an error, or
resolved with the resulting syntax tree.
run
freezes the processor if not already frozen.
function done(err[, node, file])
Invoked when transformation is complete. Either invoked with an error or a syntax tree and a file.
processor.runSync(node[, file])
Transform a syntax tree by applying plugins to it.
If asynchronous plugins are configured an error is thrown.
Node
— The given syntax tree.
runSync
freezes the processor if not already frozen.
processor.process(file|value[, done])
Process the given representation of a file as configured on the processor. The
process invokes parse
, run
, and stringify
internally.
Promise
if done
is not given. Rejected with an error or
resolved with the resulting file.
process
freezes the processor if not already frozen.
function done(err, file)
Invoked when the process is complete. Invoked with a fatal error, if any, and
the VFile
.
err
(Error
, optional) — Fatal errorfile
(VFile
)var unified = require('unified')
var markdown = require('remark-parse')
var remark2rehype = require('remark-rehype')
var doc = require('rehype-document')
var format = require('rehype-format')
var html = require('rehype-stringify')
unified()
.use(markdown)
.use(remark2rehype)
.use(doc)
.use(format)
.use(html)
.process('# Hello world!')
.then(
function(file) {
console.log(String(file))
},
function(err) {
console.error(String(err))
}
)
Yields:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
<h1>Hello world!</h1>
</body>
</html>
processor.processSync(file|value)
Process the given representation of a file as configured on the processor. The
process invokes parse
, run
, and stringify
internally.
If asynchronous plugins are configured an error is thrown.
file
(VFile
)value
(string
) — String representation of a fileVFile
— Virtual file with modified contents
.
processSync
freezes the processor if not already frozen.
var unified = require('unified')
var markdown = require('remark-parse')
var remark2rehype = require('remark-rehype')
var doc = require('rehype-document')
var format = require('rehype-format')
var html = require('rehype-stringify')
var processor = unified()
.use(markdown)
.use(remark2rehype)
.use(doc)
.use(format)
.use(html)
console.log(processor.processSync('# Hello world!').toString())
Yields:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
<h1>Hello world!</h1>
</body>
</html>
processor.data(key[, value])
Get or set information in an in-memory key-value store accessible to all phases of the process. An example is a list of HTML elements which are self-closing, which is needed when parsing, transforming, and compiling HTML.
key
(string
) — Identifiervalue
(*
, optional) — Value to set. Omit if getting key
processor
— If setting, the processor on which data
is invoked*
— If getting, the value at key
Setting information with data
cannot occur on frozen processors.
Invoke the processor first to create a new unfrozen processor.
The following example show how to get and set information:
var unified = require('unified')
console.log(
unified()
.data('alpha', 'bravo')
.data('alpha')
)
Yields:
bravo
processor.freeze()
Freeze a processor. Frozen processors are meant to be extended and not to be configured or processed directly.
Once a processor is frozen it cannot be unfrozen. New processors functioning just like it can be created by invoking the processor.
It’s possible to freeze processors explicitly, by calling .freeze()
, but
.parse()
, .run()
, .stringify()
, and
.process()
call .freeze()
to freeze a processor too.
Processor
— The processor on which freeze
is invoked.
The following example, index.js
, shows how rehype prevents
extensions to itself:
var unified = require('unified')
var parse = require('rehype-parse')
var stringify = require('rehype-stringify')
module.exports = unified()
.use(parse)
.use(stringify)
.freeze()
The below example, a.js
, shows how that processor can be used and configured.
var rehype = require('rehype')
var format = require('rehype-format')
// ...
rehype()
.use(format)
// ...
The below example, b.js
, shows a similar looking example which operates on
the frozen rehype interface. If this behaviour was allowed it
would result in unexpected behaviour so an error is thrown. This is
invalid:
var rehype = require('rehype')
var format = require('rehype-format')
// ...
rehype
.use(format)
// ...
Yields:
~/node_modules/unified/index.js:440
throw new Error(
^
Error: Cannot invoke `use` on a frozen processor.
Create a new processor first, by invoking it: use `processor()` instead of `processor`.
at assertUnfrozen (~/node_modules/unified/index.js:440:11)
at Function.use (~/node_modules/unified/index.js:172:5)
at Object.<anonymous> (~/b.js:6:4)
Plugin
unified plugins change the way the applied-on processor works in the following ways:
Plugins are a concept. They materialise as attacher
s.
move.js
:
module.exports = move
function move(options) {
var expected = (options || {}).extname
if (!expected) {
throw new Error('Missing `extname` in options')
}
return transformer
function transformer(tree, file) {
if (file.extname && file.extname !== expected) {
file.extname = expected
}
}
}
index.js
:
var unified = require('unified')
var parse = require('remark-parse')
var remark2rehype = require('remark-rehype')
var stringify = require('rehype-stringify')
var vfile = require('to-vfile')
var report = require('vfile-reporter')
var move = require('./move')
unified()
.use(parse)
.use(remark2rehype)
.use(move, {extname: '.html'})
.use(stringify)
.process(vfile.readSync('index.md'), function(err, file) {
console.error(report(err || file))
if (file) {
vfile.writeSync(file) // Written to `index.html`.
}
})
function attacher([options])
An attacher is the thing passed to use
. It configures the processor
and in turn can receive options.
Attachers can configure processors, such as by interacting with parsers and compilers, linking them to other processors, or by specifying how the syntax tree is handled.
The context object is set to the invoked on processor
.
options
(*
, optional) — Configurationtransformer
— Optional.
Attachers are invoked when the processor is frozen: either when
.freeze()
is called explicitly, or when .parse()
, .run()
,
.stringify()
, or .process()
is called for the first
time.
function transformer(node, file[, next])
Transformers modify the syntax tree or metadata of a file. A transformer is a
function which is invoked each time a file is passed through the transform
phase. If an error occurs (either because it’s thrown, returned, rejected, or
passed to next
), the process stops.
The transformation process in unified is handled by trough
, see
it’s documentation for the exact semantics of transformers.
Error
— Can be returned to stop the processNode
— Can be returned and results in further transformations
and stringify
s to be performed on the new treePromise
— If a promise is returned, the function is asynchronous, and
must be resolved (optionally with a Node
) or rejected
(optionally with an Error
)function next(err[, tree[, file]])
If the signature of a transformer includes next
(third argument), the
function may finish asynchronous, and must invoke next()
.
err
(Error
, optional) — Stop the processnode
(Node
, optional) — New syntax treefile
(VFile
, optional) — New virtual filePreset
Presets provide a potentially sharable way to configure processors. They can contain multiple plugins and optionally settings as well.
preset.js
:
exports.settings = {bullet: '*', fences: true}
exports.plugins = [
require('remark-preset-lint-recommended'),
require('remark-comment-config'),
require('remark-preset-lint-markdown-style-guide'),
[require('remark-toc'), {maxDepth: 3, tight: true}],
require('remark-github')
]
index.js
:
var remark = require('remark')
var vfile = require('to-vfile')
var report = require('vfile-reporter')
var preset = require('./preset')
remark()
.use(preset)
.process(vfile.readSync('index.md'), function(err, file) {
console.error(report(err || file))
if (file) {
vfile.writeSync(file)
}
})
unified is built by people just like you! Check out
contributing.md
for ways to get started.
This project has a Code of Conduct. By interacting with this repository, organisation, or community you agree to abide by its terms.
Want to chat with the community and contributors? Join us in Gitter!
Have an idea for a cool new utility or tool? That’s great! If you want
feedback, help, or just to share it with the world you can do so by creating
an issue in the unifiedjs/ideas
repository!
Preliminary work for unified was done in 2014 for
retext and inspired by ware
. Further incubation
happened in remark. The project was finally externalised
in 2015 and published as unified
. The project was authored by
@wooorm.
Although unified
since moved it’s plugin architecture to trough
,
thanks to @calvinfo,
@ianstormtaylor, and others for their
work on ware
, which was a huge initial inspiration.
FAQs
parse, inspect, transform, and serialize content through syntax trees
The npm package unified receives a total of 8,380,581 weekly downloads. As such, unified popularity was classified as popular.
We found that unified demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
ESLint has added JSON and Markdown linting support with new officially-supported plugins, expanding its versatility beyond JavaScript.
Security News
Members Hub is conducting large-scale campaigns to artificially boost Discord server metrics, undermining community trust and platform integrity.
Security News
NIST has failed to meet its self-imposed deadline of clearing the NVD's backlog by the end of the fiscal year. Meanwhile, CVE's awaiting analysis have increased by 33% since June.